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REVISED APPEAL BRIEF PURSUANT TO 37 C.F.R. §§ 41.31 AND 4137 

This Revised Appeal Brief is being filed in response to a Notice of Non-Compliant 
Appeal Brief mailed on October 15, 2007. In this notice, the Patent Office objected to the 
description of the status of the claims for being unclear and possibly including a typographical 
error. To clarify this section and remedy any errors, the following revised appeal brief includes 
an expanded explanation of the status of the claims and adds a discussion of the status of claim 2 

1. REAL PARTY IN INTEREST 

The real party in interest is Synoptics Limited, the Assignee of the above-referenced 
application by virtue of the Assignment recorded at reel 012446, frame 0787, and dated 
January 2, 2002. 



2. RELATED APPEALS AND INTERFERENCES 

Appellant is unaware of any other appeals or interferences related to this Appeal. The 
undersigned is Appellant's legal representative in this Appeal. Synoptics Limited will be 
directly affected by the Board's decision in the pending appeal. 

3. STATUS OF CLAIMS 

Claim 1 is pending and under final rejection. Claim 2 is pending and objected to as 
allowable but for its dependence on claim 1 . Claim 3 is a multiple-dependent claim that depends 
in the alternative from claim 1 and from claim 2. Cf M.P.E.P. § 608.0 l(n) (explaining the 
proper format for multiple-dependent claims). Claim 3/1 — claim 3 with direct dependence from 
claim 1 — is pending and under final rejection. Claim 3/2 — claim 3, with dependence from claim 
2 — is objected to as allowable but for its dependence on claim 1 through claim 2. The rejection 
of claims 1 and 3/1 is being appealed. 

4. STATUS OF AMENDMENTS 

The present application is not subject to any pending amendments. 

5. SUMMARY OF CLAIMED SUBJECT MATTER 

This appeal addresses independent claim 1 . Below, Appellant explains this claim by 
identifying specific embodiments in the specification. While these embodiments exemplify the 
appealed claim, they do not necessarily define its scope. Thus, this claim should not be 
construed as limited to the following embodiments by virtue of this explanation. 
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Independent claim 1 recites: 



A method of creating an image which includes the steps of: 

obtaining a substantially linear representation of the 
brightness of an image, the method comprising, for each of a set of 
pixels (x, y) in a two dimensional array, calculating an estimate of 
the true image intensity (i xy ) as a weighted average of n samples of 
the apparent image intensity (v n>xy ) as 



n ± n 

n n 



where v n>xy is the apparent intensity measured, n is greater than or 
equal to 2, T n is the exposure time, K is the gain of the system, C is 
an offset and w n>xy is a weighting factor which is defined to 
maximise the signal to noise ratio and discard insignificant, that is 
saturated or near zero, values; 

thereafter saving each of the values i xy together with other 
data representing the image; and 

outputting the image to a display or to a printing device. 



In the application, the subject matter of claim 1 is exemplified by a process performed by 
software executed by a computer 3. See application, paragraph 46; and FIG. 1. The process 
includes performing the following steps to obtain a substantially linear representation of the 
brightness of an image: 

1) Initially, the process acquires two or more images of the same object. See 
application, paragraph 56. Each of the images is acquired with a different 
exposure time. So even if the camera 1 has a limited dynamic range, the 
images with a longer exposure time capture detail from darker areas of the 
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object, and the images with a shorter exposure time capture detail from 
brighter areas of the object. See id. Consequently, the images as a group 
depict a wider range of light intensity than any one image in the group. See 
id. at paragraph 4. In this example, the number of images exemplifies the 
referent quantity of the variable "n" in claim 1, so for two-input images, n 
equals 2. 

2) Next, the exemplary process combines the two or more input images into a 
single output image that shows the full range of light intensity captured by all 
of the images. See id. at paragraph 56. That is, details from both the dark 
regions and the light regions are contained in the single output image. See id. 
at paragraph 2. The exemplary process combines the images pixel-by-pixel, 
so one pixel from each input image is combined to form a single pixel in the 
resulting image. See id. paragraph 12. Thus, in this example, if n equals 2, a 
pixel from one input image is combined with a pixel from the other input 
image to calculate the intensity of a single pixel in the output image. This 
process is repeated for each of the pixels in the output image. As explained 
below, the input pixels are combined according to an equation that produces a 
substantially linear representation of object brightness. See id. at paragraph 
19. That is, the brightness of the representation is substantially proportional to 
the brightness of the object being imaged, so neither the darker areas nor the 
lighter areas of the object are saturated in the representation. See id. 
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In this embodiment, the parameters of the pixels in both the input images and output 
image exemplify the referent quantities of some of the variables in claim 1 . The referent 
quantity of the variable "i" in claim 1 is exemplified by the intensity of a pixel in the output 
image, and the referent quantities of the subscript variables "x" and "y" that index the variable 
"i" are exemplified by pixel locations in the resulting image, e.g., the row and column addresses 
of the pixel. See id. at paragraph 12. Similarly, the referent quantity of the variable "v" is 
exemplified by the pixel intensities in each of the two or more input images. The variable v is 
indexed by both the subscript variable "n," which in the present example refers to the number of 
input images, and the subscript variables x and y, referring in some instances to row and column 
locations. See id. 

Examples of the other variables in the equation of claim one are also described in the 
application. "T n ," the exposure time for an input image, is discussed both in paragraph 4 and in 
paragraph 21 , which describes considerations that may be relevant when selecting values for 
"T n ." Examples of the gain of the system, "K," are described in paragraphs 7, 13, 40, and 41 of 
the application. Examples of the offset, "C," are described in paragraphs 7, 13, 23-28, and 56 of 
the application, and an example of the weighting factor, "w," is described in paragraph 13. In 
this embodiment, the variable "w" is indexed both by the subscript variable "n," exemplified by 
an index number of the input pictures, and the subscript variables "x" and "y," exemplified by 
row and column locations of pixels in the input images. 

The referent quantities of these variables are combined through the equation of claim 1 . 
The combination produces a digital image whose values are related linearly (i.e. proportionally) 
to the intensity of light falling on the light sensor (e.g., the camera 1 in FIG. 1), even when there 
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is a very wide range of such intensities. See id. at paragraph 19. The equation combines the 
series of images taken with different exposure times by both thresholding data in the input 
images to remove the effects of saturation and applying the appropriate weighing factor to each 
pixel of each input image. See e.g., id. at paragraphs 16-19. 

Examples of the other steps of claim 1, saving and outputting, are described in paragraph 
44: saving is exemplified by the act of placing the digital image 4 in the computer's memory, 
and outputting is exemplified by the act of displaying the digital image 4 on a monitor 5. See id. 
at paragraph 44; and FIG. 1 . 

6. GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

The Examiner rejected claims 1 and 3 under 35 U.S.C. § 103 (a) as being obvious over 
U.S. Patent 5,828,793 (Mann) in view of U.S. Patent 4,590,582 (Umemura). The Examiner also 
rejected claim 2 for relying on claim 1 but indicated that it would be allowable if placed in 
independent form, and Appellant reserves the right to do so if necessary. Appellant respectfully 
urges the Board to reverse these rejections. 

7. ARGUMENT 

To reject the pending claims, the Examiner misapplied long-standing and binding legal 
precedents. Below, Appellant summarizes both the rejection and the relevant precedent and, 
then, explains why the rejection violates the relevant precedent. 
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The Rejection 

In the Final Office Action, the Examiner rejected claims 1 and 3/1 as being obvious over 
Mann in view of Umemura, remarking: 



Mann discloses a method of creating an image with a still 
video camera (col. 1 1 lines 43-46, figure 8, element 202). Mann 
further teaches that the image is transferred to a computer to be 
stored on a main memory 210 represented as 212i, 2122, 2133 etc. 
(col. 1 1 lines 46-54). Mann also teaches that the composite images 
[are] formed from a series of input images wherein every pixel of 
the composite image is drawn from the corresponding pixel in each 
of the input source images according to a weighted average. The 
weighting is based on a certainty function associated with each 
source image pixel corresponding to an output pixel in the final 
composite image. The value of the relevant pixel parameter for a 
given final-image pixel (weighted average of n samples) is given 
by 



where c n is the certainty function associated with the 
corresponding pixel of each source image n (col. 6 line 5 1-col 7 
line 8). It is noted that P n (pixel parameter) is dependent upon 
exposure time, brightness or luminance and the gain of the system. 
Mann teaches that the resulting pixel image represented by the 
expression above is saved in a target buffer 250 whose contents are 
shown on screen display 234 (col. 12 lines 32-49). The features 
such as gamma correction (other image data) are also stored in the 
target image data (col. 13 lines 4-8). 

Mann fails to teach explicitly obtaining a substantially 
linear representation of the image by combining two images. 
However Umemura teach that when various parameters are used 
for one original image data, the image data filtered by the 3.times.3 
filter 2 and stored in the memory 3 is read out for each such 
parameter. Then, the original image data and the filtered image 
data are subjected to linear combination by the adder device 5 in 
order to achieve high-speed processing (col. 8 lines 23-39, figure 
9, also see col. 7 line 53-col. 8 line 23, figure 1). 
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Therefore taking the combined teachings of Mann and 
Umemura, it would be obvious to one skilled in the art at the time 
of the invention to have been motivated to have obtained a 
substantially linear representation of the image by summing two 
images in order to achieve high- speed processing as taught in 
Umemura (col. 8 lines 23-39). 

[Claim 3/1] 

Mann teaches that the different images are color so that the 
offset will be color dependent (col. 13 lines 21-30). 

Final Office Action, pages 3 and 4. 

As explained below, this rejection fails to satisfy the Examiner's burden to demonstrate 
that all of the elements are obvious in view of the prior art. Specifically, the rejection is flawed 
because the Examiner has not proved that one of ordinary skill in the art would find it obvious to 
obtain a substantially linear representation of the brightness of a scene. 

Legal Precedent 

The burden of establishing a prima facie case of obviousness falls on the Examiner. Ex 
parte Wolters andKuypers, 214 U.S.P.Q. 735 (PTO Bd. App. 1979). Obviousness cannot be 
established by combining the teachings of the prior art to produce the claimed invention absent 
some teaching or suggestion supporting the combination. ACS Hospital Systems, Inc. v. 
Montefiore Hospital, 732 F.2d 1572, 1577, 221 U.S.P.Q. 929, 933 (Fed. Cir. 1984). Even if, ad 
arguendo, the references could be combined or modified in the proposed manner, it does not 
render the resultant combination obvious unless the prior art also suggests the desirability of the 
combination. See In re Mills, 916 F.2d 680, 16 U.S.P.Q.2d. 1430 (Fed. Cir. 1990). Accordingly, 
to establish a prima facie case, the Examiner must not only show that the combination includes 
all of the claimed elements, but also a convincing line of reason as to why one of ordinary skill 
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in the art would have found the claimed invention to have been obvious in light of the teachings 
of the references. Ex parte Clapp, 227 U.S.P.Q. 972 (B.P.A.I. 1985). 

It is Not Obvious to Obtain a Substantially Linear Representation of the Brightness of a 
Scene by Combining Two or More Images 

This rejection is flawed because neither cited reference teaches obtaining a substantially 
linear representation of the brightness of a scene by combining two or more images. Indeed, the 
Examiner has recognized that the primary reference, Mann, does not teach obtaining such a 
representation. See Final Office Action, page 4. The rejection instead relies on Umemura for 
this feature, but as is explained below, Umemura does not teach or suggest obtaining a 
substantially linear representation of the brightness of a scene either. 

Umemura teaches a technique that distorts image brightness to enhance edges. The edges 
are refined, according to Umemura, by spatially filtering an image — that is, changing pixel 
brightness based on neighboring pixels — and combining the filtered image with a version of the 
original image. See Umemura, col. 2, lines 49-66; and FIG. 1 (illustrating an adder device 6 that 
combines the original image, stored in OID memory 1, with the filtered image, stored in FID 
memory 3). The combination is depicted in FIG. 6 of Umemura and described in column 6, lines 
43-52. As explained in this passage, the filtered image data is added to the original image data in 
the adder 76, after both sets of image data are multiplied by weighting coefficients in multiplier 
elements 65 and 66. See id. Thus, an original source image is combined with a filtered version 
of itself to produce an output image that exaggerates certain image features, such as edges. 
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Although Umemura refers to the combination operation as a "linear combination," it does 
not produce a substantially linear representation of image brightness. Umemura combines the 
original image with a filtered image that distorts image brightness. To produce the filtered 
image, Umemura combines pixels from different locations in the single, original source image. 
See Umemura, col. 1, lines 17-25. The filter is described as a 3x3 linear non-recursive type two- 
dimensional filter, and Umemura explains that it is effective "for noise removal, sharpness 
control, edge enhancement and the like." See id. Thus, Umemura teaches sharpening or blurring 
the original image through a spatial filter that changes pixel intensity according to adjacent pixel 
intensities. Consequently, the intensity of a pixel i x , y in the filtered image is not proportional to 
image brightness at a location corresponding to x and y; it is a function of other pixels nearby. It 
follows that, regardless of whether the combination of the original image with the filtered image 
is a linear operation, the combination does not produce a substantially linear representation of 
image brightness , because the spatially-filtered image is, by definition and design, distorted. 

Moreover, this inadequacy in the rejection is not cured by the combination of references. 
Combining the teaching of Mann and Umemura would not produce an image whose values were 
linearly related to the light intensity. In fact, the calculation step of Umemura would just blur or 
sharpen each of the individual source images. 

Mann teaches how to calculate an output image, each of whose pixels are calculated by 
combining the corresponding pixels from each of a number of source images. As the Examiner 
has already accepted, Mann does not teach how to make each pixel's value proportional to the 
light intensity at the pixel location. Mann does not teach how to derive the weights so as to 
result in a linear relationship between light intensity and digital value. Mann simply combines a 
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number, of images taken at different exposures. Mann does not discuss altering the spatial 
characteristics of the image, which would be achieved by combining pixels taken from different 
locations. 

As explained, Umemura teaches how to achieve a user-controlled modification of the 
spatial frequency characteristics of a single source image. Umemura does not discuss a process 
that combines a number of source images. The technique of Umemura combines pixels from 
different locations in the single source image. 

To combine the teaching of the two documents is non-obvious. In the unlikely event of a 
skilled person considering combining the teachings from the two documents there are two ways 
in which this might be done: 

1) The source images could be spatially filtered according to Umemura and then 
processed according to Mann. The spatial filtering, however, would destroy 
the relationship between light intensity and pixel value within each source 
image that is essential for Mann's algorithm. Further, the final result would be 
spatially filtered and therefore not reproduce faithfully the frequencies of the 
source images. Finally, it would not provide a result image in which each 
pixel's value is proportional to the light intensity at the pixel location. 

2) The source images could be combined into a single image according to Mann 
and then spatially filtered according to Umemura. In this case each method 
would 'work' independently within its own terms of reference. However, it 
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would not provide a result image in which each pixel's value is proportional to 
the light intensity at the pixel location. 

Accordingly, it clear that is not obvious to combine the teaching from the two documents, 
and, even if the documents were combined, it is not possible to arrive at the result of the present 
invention. 

As has been repeatedly explained to the Examiner, the present invention, like Mann's, 
combines only spatially corresponding pixels from each source image to calculate each pixel in 
the result. To do otherwise (that is, to combine pixels from other locations) would destroy the 
information necessary to perform the calculation correctly. (For example, if a 3x3 spatial filter 
were used to combine neighbouring pixels and some of those pixels were 'invalid' because they 
were saturated, it would not be possible to determine whether such saturation had taken place). 
The present invention differs from Mann's, however, in that it teaches how to combine the pixels 
in order that each output pixel is directly proportional to the light intensity at the pixel location. 
The fact that Umemura teaches a method in which a result is formed by a weighted sum 
(otherwise referred to as a linear combination') of two other images - a step that must appear in 
just about every digital image processing operation ever designed - does not mean that it 
provides any teaching on how to achieve an output image whose pixels are directly proportional 
to light intensity. 

After the Appellant alerted the Examiner to this missing feature, the Examiner responded 
by simply reiterating that Umemura teaches a linear operation and begging the question of 
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whether the operation can succeed in producing an image that is linearly related to the pattern of 
light falling on the sensor. In the Final Office Action, the Examiner asserted: 



Applicant argues at pages 8 and 9 that the present invention 
differs from Mann and Umemura in that it teaches how to combine 
the pixels in order that each output pixel is directly proportional to 
the light intensity at the corresponding pixel location. To do 
otherwise (that is, to combine pixels from other locations) would 
destroy the information necessary to perform the calculation 
correctly. 

In response, the Examiner respectfully disagrees with the 
Applicants' assessment of the claims since the claims are written 
much broader than the Applicants' arguments. It appears that the 
Applicants brought [the] specification into the claims which are 
written much broader than what they should be. 

In fact, Umemura teach that when various parameters are 
used for one original image data, the image data filtered by the 
3.times.3 filter 2 and stored in the memory 3 is read out for each 
such parameter. Then, the original image data and the filtered 
image data are subjected to linear combination by the adder device 
5 in order to achieve high-speed processing (col. 8 lines 23-39, 
figure 9, also see col. 7 line 53-col. 8 line 23, figure 1) which reads 
on the claimed limitations "obtaining a substantially linear 
representation of the image by combining two images". 

Therefore, in response to applicant's argument that the 
references fail to show certain features of applicant's invention, it 
is noted that the features upon which applicant relies (i.e., how to 
combine the pixels in order that each output pixel is directly 
proportional to the light intensity at the corresponding pixel 
location) are not recited in the rejected claim(s). Although the 
claims are interpreted in light of the specification, limitations from 
the specification are not read into the claims. See In re Van Geuns, 
988 F.2d 1 181, 26 USPQ2d 1057 (Fed. Cir. 1993). 

Final Office Action, pages 2 and 3. 

The Examiner asserts that claim 1 is broader than the Appellant's previous arguments 
because Appellant argued that claim 1 recites obtaining a substantially linear representation of 
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the brightness of a scene by combining corresponding pixel locations. The Examiner's assertion, 
however, is false. The claims include formulae where every pixel value explicitly contains the 
suffix xy to denote that the result at the two-dimensional coordinates (x,y) are derived only from 
pixel values with the same suffix and constants. Therefore, each pixel result is obtained without 
any influence of neighbouring pixels. If the formulae referred at all to neighboring pixels , the 
suffices would include terms like x+l,y or x, y-1 ; they do not. 

Further, the Examiner's argument ignores the phrase "of the brightness of an image." 
Indeed, in the rejection, the Examiner omits the word "brightness," and none of the Office 
Actions allege that the cited art produces a representation that is substantially linear with respect 
to the brightness of an image . 

The Examiner still has not explained how the cited references teach "obtaining a 
substantially linear representation of the brightness of an image." As explained above, Umemura 
teaches spatially filtering an image and then combining the spatially-filtered image with the 
original image. Regardless of whether the combination operation is linear, it does not produce a 
substantially linear representation of scene brightness, because the input to the process has been 
subject to a non-linear operation: very bright and very dark pixels are distorted non-linearly by 
the camera due to its limited dynamic range. 

Information lost in this non-linear operation cannot be restored by spatial filtering 
technique such as Umemura's; it can only be regained by combining two or more images that 
cover different parts of the total dynamic range of the scene. Indeed, the intended purpose of the 
system taught by Umemura is to enhance features, such as edges, in the image by spatially 



Page 14 



filtering the image and exaggerating these features; it cannot contribute to the requirement to 
'undo 1 the camera's non-linear response (as the present invention does). See Umemura, col. 1, 
lines 39-52. In short, Umemura's pixels distort image brightness. The Examiner has not 
identified a reference that describes the claimed type of linear operation, and the present 
rejection fails to satisfy the Examiner's burden of proof. 

For these reasons among others, Appellant respectfully requests withdrawal of the 
rejection under 35 U.S.C. § 103 and allowance of all pending claims. 



Page 15 



» 



Conclusion 

In view of the above remarks, Appellant respectfully submits that the Examiner has no 
supportable position or evidence that claims 1-3 are obvious under Section 103. Accordingly, 
Appellant respectfully requests that the Board find claims 1-3 patentable over the art of record, 
withdraw all outstanding rejections, and allow claims 1-3. 

In accordance with 37 C.F.R. § 1.136, Appellant requests that this and any future reply 
requiring an extension of time be treated according to the General Authorization For Extensions 
Of Time previously submitted. 

The Commissioner is authorized to charge the requisite fee of $500.00, and any 
additional fees which may be required, to Deposit Account No. 06-1315, Order No. GJEL:0003 / 
FLE. 



Respectfully submitted, 



Date: November 7, 2007 




Michael G. Fletcher 
Reg. No. 32,777 
Fletcher Yoder 
FLETCHER YODER 
P.O. Box 692289 
Houston, TX 77269-2289 
(281)970-4545 
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8. APPENDIX OF CLAIMS ON APPEAL 

1 . (Previously presented) A method of creating an image which includes the steps 



obtaining a substantially linear representation of the brightness of an image, the method 
comprising, for each of a set of pixels (x, y) in a two dimensional array, calculating an estimate 
of the true image intensity (i xy ) as a weighted average of n samples of the apparent image 
intensity (v n ,xy) as 



where v n , xy is the apparent intensity measured, n is greater than or equal to 2, T n is the exposure 
time, K is the gain of the system, C is an offset and w n ,xy is a weighting factor which is defined to 
maximise the signal to noise ratio and discard insignificant, that is saturated or near zero, values; 
thereafter saving each of the values i xy together with other data representing the image; 

and 

outputting the image to a display or to a printing device. 

2. (Original) A method according to claim 1, wherein a linear relationship is 
established between images recorded with different exposure times by the use of a perpendicular 
regression technique whereby each image is transformed to match the scale and offset of the first 
in the series and whereby the weighted average is calculated as: 



of: 




n 



n 
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n 



l X y — 



n 



njcy 



where a n and b n are the gradient a and offset b measured between image n and image n 
(ai=l; bi=0) when 



0 when 
0 



v min < v n,xy < v max 



3. (Original) A method according to claim 1 or claim 2, wherein the image is a 
coloured image and the offset is colour dependent. 
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EVIDENCE APPENDIX 

None 



RELATED PROCEEDINGS APPENDIX 

None 
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